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Abstract 

Whether or not deconfined quark phase exists in neutron star cores 
and represents the ground state of neutral matter at moderate densities 
are open questions. We use two realistic effective quark models, the three- 
flavor Nambu-Jona-Lasinio model and the modified quark-meson coupling 
model, to describe the neutron star matter. After constructing possible 
hybrid equations of state (EOSes) with unpaired or color superconduct- 
ing quark phase, we systematically discuss the observational constraints 
of neutron stars on the EOSes. ft is found that the neutron star with 
pure quark matter core is unstable and the hadronic phase with hyper- 
ons is denied, while hybrid EOSes with two-flavor color superconducting 
phase or unpaired quark matter phase are both allowed by the tight and 
most reliable constraints from two stars Ter 5 1 and EXO 0748-676. And 
the hybrid EOS with unpaired quark matter phase is allowed even com- 
pared with the tightest constraint from the most massive pulsar star PSR 
J075f+f807. Therefore, we conclude that the ground state of neutral 
matter at moderate densities is in deconfined quark phase likely. 

PACS number(s): 12.38. Mh, 12.39.-x, 26.60.+C 



1 INTRODUCTION 

Neutron stars are some of the densest objects in the universe and the density 
in the inner core of a neutron star could be as large as several times nuclear 
saturation density (= 0.17 fm -3 ) pQ. The core of neutron star is so dense that 
phase transition from confined hadronic phase to deconfined quark phase may 
exist. The possible emergence of deconfinement phase in neutron star cores has 
aroused great interests since it may have a distinct effect on the neutron star 
structure^]- Up to now, however, no conclusive observational or experimen- 
tal evidence suggests that the quark matter core conjecture is true and this still 
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remains an open question. In our recent work where the hadronic phase contain- 
ing octet baryons was considered, it was found that the equation of state (EOS) 
with deconfinement phase would be ruled out by the observational mass limit of 
1.68M© of star Ter 5 I [5]. Similar result was also educed by Ozel who reported 
that EOSes with exotic phases would be ruled out by the inferred mass and ra- 
dius for star EXO 0748-676 and she concluded that the ground state of matter 
was hadrons and not deconfined quarks[4]. While Alford et al. compared Ozel's 
observational limits with predictions based on a more comprehensive set of pro- 
posed equations of state from the literature, and concluded that the presence of 
quark matter in EXO 0748-676 was not ruled ouL5j. Therefore, the existence 
of deconfined quark phase in neutron star cores and the ground state of neutral 
matter at moderate densities are controversial. Our goal of the present paper is 
to systematically investigate the observational constraints on the deconfinement 
in neutron star cores and check whether deconfined quarks are possible to exist 
in the ground state of neutral matter at moderate densities. 

It is expected that at extreme conditions chiral symmetry can be restored 
and quarks and gluons become deconfined [BJ. Consequently, quark matter was 
frequently dealt with as a non- interacting quark gas, i.e., unpaired quark matter 
(UQM), and usually described by a phenomenological bag model[7]- According 
to the BCS theory [5], any attractive interaction in a cold fermi sea will cause 
Cooper instability in the vicinity of fermi surface in the momentum space and su- 
perconductor will be formed. Because of the attractive quark-quark interaction 
in the color antitriple channel[9] it is expected that color superconducting state, 
with a spontaneous breakdown of the non-Abelian SU(3) color gauge group, 
would be the ground state of quark matter. The color superconducting state 
has attracted great interests since it was found that superconducting gap could 
be ~ lOOMeV due to the nonperturbative features [ID]. Depending on quarks 
participating in a diquark condensation, one can distinguish several color super- 
conducting phases. The prominent two are the two-flavor color superconducting 
(2SC) phase containing ud pairs together with unpaired strange quarks and the 
color-flavor locked (CFL) phase containing ud, ds and us pairs(for recent reviews 
seeRef. [IT]). 

We adopt two realistic effective quark models for quark and hadronic phases 
respectively to describe the neutron star matter. The extended three-flavor 
Nambu-Jona-Lasinio (NJL) model with determinant interaction(12|. which shares 
many symmetries with QCD, is used to calculate the properties of quark mat- 
ter in UQM, 2SC and CFL phases. For hadronic phase, we adopt the EOSes 
of neutron-proton matter and hypernuclear matter with octet baryons by the 
improved modified quark-meson coupling (MQMC) model[3l [13]. The MQMC 
model gives satisfactory description for saturation properties of nuclear matter [T4] 
and could reproduce the bulk properties of finite nuclei wellQ~5]. We then as- 
sume a sharp (first order) transition from pure hadronic to pure quark phase 
and consider only the homogeneous quark phase and not mixed phase between 
different quark phases. 

The outline of this paper is as follows: In Sec. [2] we briefly introduce 
the three-flavor NJL model for dense quark matter and the MQMC model for 
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hadronic phase. In Sec. G2 we construct the possible equations of state of neutron 
star matter and then discuss the observational constraints on the EOSes. Sec. 
[3] is devoted to summaries and conclusions. 

2 THE MODEL 

2.1 QUARK PHASE 

Strange quark matter at moderate densities can be effectively described by the 
three- flavor NJL model [TH [T7]. The Lagrange density of extended three- flavor 
NJL model with six-fermion determinant interaction (t'Hooft term) is given by 

£njl = q (iY*^ - m ) q + C qq + C qq , (I) 

where 

8 

-K [detf {q(l + 7 5 )tf) + det/ (g(l - l5 )q)} (2) 

and 

^ii = H X! {qil5TA>^A>q c ) {q c ilsT A \ A ,q) . (3) 

A=2,5,7 J 4'=2,5,7 

Here, q = (u,d,s) T denotes the quark fields with three colors. And the cur- 
rent quark mass matrix has the form rho = diag(mo u , mod-, m-Os) hi the flavor 

space, where mo u = mod — mriQq is assumed throughout this paper. tq=-J^])^ 
is proportional to the unit matrix in the flavor space, ta and X A (A=l,. . . ,8) 
are Gell-Mann matrixes in flavor and color spaces respectively. q c = Cq T is the 
charge-conjugate spinor. 

In the present work, we restrict ourselves to bulk quark matter in mean-field 
approximation and focus on the chiral condensates defined as 

0/ = f = u,d,s, (4) 

and the three-flavor diquark condensates being 

A A = -2H (q c j 5 T A X A q) , A = 2,5,7. (5) 

After bosonization, one obtains the linearized version of the model in the mean- 
field approximation, 

£= qii^dft - m)q 

+ \ \S ( A a7sTaAa) q c + q c {-A* a j 5 t a X a ) q] 

1 A 

-iwEi^i 2 - 2G E^ +4ir ^ ( 6 ) 

A f 
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where we have introduced the constituent quark mass 
mou - 4G</> n + 2K4> d 4> s 

m od - 4G</> d - 2Kcj) s <p u 



m 0s - AG(p s - 2Kcj) u (j) d 



Employing Nambu-Gorkov formalism, then the thermodynamic potential per 
unit volume at temperature T is obtained via the finite temperature field theory [18] 
and takes the form, 
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= 2G J2 

f=u,d,s 
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(9) 



(10) 



is the contribution from the electron gas with chemical potential \x e . 

In Eq.([8"]). u>i is the energy of a quasiparticle and can be obtained by diag- 
onalizing the inverse propagator in Nambu-Gorkov basis. Or, equivalently, tUi 
can be obtained by calculating the eigenvalues of the 72 x 72 matrix 
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And the chemical potential operator p, is a diagonal 9x9 matrix in flavor and 
color space. By introducing the quark number chemical potential /_t, electro- 
chemical potential /iQ and two additional chemical potentials /13 and /is coupled 
to the color charges A3 and As respectively, jl can be expressed as(see Ref. [TB] 
for details) 



In beta equilibrium, we have 



2\/3 



Me = -Mq. 



(12) 



(13) 



The order parameters, 0/ and A^, can be obtained by minimizing the ther- 
modynamic potential, and are equivalently given by the gap equations, 



d<f> f 

on 



= 0, f = u,d,s, 
= 0, A = 2, 5, 7. 



(14) 
(15) 
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With the diquark condensates, we can explicitly distinguish different quark 
phases as: 



A 2 = A 5 = A 7 = : UQM; 
A 5 = A 7 = 0, A 2 ^ : 2SC; 
A 2 ,A 5 ,A 7 ^0: CFL. 

Dense quark matter in neutron star is electrical and color neutral, then the 
following relations should be maintained: 

M) 

ns = -^ = Q, (17) 

n 8 = ~ = 0. (18) 
d/is 

Finally, for a given quark number density 

dfl 

the energy density £ and pressure V at zero temperature are given by: 

£ = n + Q vac + fin, (19) 

v = -n- n vac . (20) 

r2 vac is chosen so that V and £ vanish in vacuum, which is the only way to 
uniquely determine the EOS within the NJL model without any further as- 
sumption. 

For the model parameters, we take the values as follows. To regularize 
the divergent integrals we need a sharp cutoff A in 3-momentum space since the 
NJL model is nonrenormalizable. Thus we have a total of 6 parameters, namely, 
the current masses mo s and mo, for strange and nonstrange quarks, the three 
couplings G, K and H, and the cutoff A. Following the method adopted in 
Ref.pjj], we get A=602.8MeV, GA 2 =1.803, J ftTA 5 =12.93 and m Os =140.9MeV by 
fitting the meson masses 20 = 134.98MeV, ttik = 497.65MeV and m n i = 
957.78MeV and the n decay constant f„ = 92.2MeV[2"T] while mo g is fixed at 
5.5MeV. H = G is set as has been used in Ref.|16). 

2.2 HADRONIC PHASE 

For the hadronic phases, we focus on the neutron-proton matter and the hy- 
pernuclear matter consisting of the baryon octet, i.e., p, n, A, E + , S°, S° 
and S - . The hadronic phases are described with the improved MQMC model 
which has been discussed in detail in Ref.[3]- In order to make the calculation 
for hadronic phases compatible with that for quark phases, we should take the 
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Table 1: The new zero-point motion parameters and radii for the modified 
quark-meson coupling model. 





M(MeV) 


Z 


R(fm) 


N 


939.0 


2.0403 


0.6000 


A 


1115.7 


1.8099 


0.6459 


£+ 


1189.4 


1.6318 


0.6722 


S° 


1192.6 


1.6236 


0.6733 


£" 


1197.4 


1.6114 


0.6749 


S° 


1314.8 


1.4743 


0.6922 




1321.3 


1.4567 


0.6942 



Table 2: New independent coupling constants, which have been defined in 
Ref.[3]. 



9a ,d 


9T 






0.9685 


2.7071 


7.9288 


6.8732 



same current quark masses in both models. Therefore the current quark masses 
take mou = mod — 5.5MeV and mo s =140.9MeV, which are different from those 
used in Ref.[3], and the other inputs, such as the mass spectrum and the nu- 
cleon's radius, are unchanged. As a consequence, the bag constant in vacuum 

1 /4 

is changed to be B Q =187.7716MeV. The obtained zero-point motion parame- 
ters and bag-radii for baryons and coupling constants are all changed, their new 
values are listed in TABLE [1] and TABLE O respectively. 

3 RESULTS AND DISCUSSIONS 
3.1 EQUATION OF STATE 

First order phase transition takes place at critical point where baryon chemical 
potentials and pressures for the two phases are equal. The pressures of hadronic 
and quark phases are shown in Figure [T] as a function of baryon chemical po- 
tential and the position of hadron-quark phase transition can be easily read off 
as the point where the lines V^b) cross. From the figure, we can find four 
possible hadron-quark phase transitions, i.e., A, B, C and D. EOS of the hyper- 
nuclear matter (npH) has no cross with those of quark phases 2SC or UQM, and 
has only one cross at A with that of CFL. Whereas the neutron-proton matter 
(np) could change into deconfined quark phases in UQM, 2SC or CFL state at 
B, C and D. Therefore, we could construct totally four kinds of hybrid EOSes 
between hadronic phase and quark deconfinement phase. 
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Figure 1: Pressure of hadronic or quark phase as a function of baryon chemical 
potential, np (black) denotes neutron-proton matter and npH (red) denotes the 
hypernuclear matter consisting of octet baryons. UQM (green), 2SC (magenta) 
and CFL (blue) denote the three kinds of homogeneous quark phases. Squares 
ordered by A, B, C and D mark the hadron-quark phase transitions and E is 
the phase transition between 2SC and CFL phases. Values in parentheses are 
the critical chemical potentials. 

EOSes are plotted in Figure [5] and possible phase transitions are also shown 
by dotted lines marked by A, B, C, D and E. The hybrid EOS of npH+CFL can 
be constructed by combining the EOS of pure hypernuclear matter npH and the 
EOS of pure quark matter in CFL state for low and high pressures respectively, 
with a hadron-quark phase transition at A. The other EOSes of np+UQM, 
np+2SC and np+CFL could be similarly constructed. There are some notable 
features for the hybrid EOSes. First, the hadron-quark phase transitions are 
first order, which is an inevitable result because we have chosen a sharp interface 
as claimed in Sec. [T] Second, the energy gaps of phase transitions are rather 
large, which is found to have profound effect on the star structure and is going 
to be discussed. Among the hybrid EOSes, the stiffest one is of np+UQM, and 
the next are of np+2SC, np+CFL and npH+CFL in turn. 

3.2 MAXIMUM MASS 

By solving the Tolman-Oppenheimer-Volkoff equations [22], mass-radius rela- 
tions are obtained for different equations of state, which are given in Figure [3l 
The softest EOS of npH+CFL predicts a maximum mass of 1.54 M Q . Com- 
pared with the best measured pulsar mass 1.44M Q in the binary pulsar PSR 
1913+16 [33], which had been taken as the lower limit of neutron star's maxi- 
mum mass for many years, all the EOSes here are allowed. However, very recent 
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Figure 2: Energy vs. pressure for equations of state. The solid lines represent 
the EOSes of pure phases. And the thin dotted lines indicate the first order 
phase transitions. A, B, C, D and E are the same as those defined in Figure [T] 

measurements strongly indicate that there are more massive neutron stars. The 
typical neutron star is reported by Ransom et al. who inferred that at least one 
of the stars in Terzan 5, the Ter 5 I, is more massive than 1.48, 1.68, or 1.74 M Q 
at 99%, 95%, and 90% confidence levels 24 . Therefore, as indicated in Figure 
02 imposing the tighter observational constraint of 1.68M Q at 95% confidence 
level, pure hadronic EOS with hyperons and the hybrid EOS of npH+CFL are 
firmly ruled out, while the hybrid EOSes of np+UQM, np+2SC and np+CFL 
can be compatible with this constraint. So far, the most massive pulsar star 
reported is PSR J0751+1807, which has a inferred mass of 2.1±O.2M covering 
68% confidence uncertainties }25| . Therefore, PSR J0751+1807 gives the tightest 
mass constraint of 1.9M at about 68% confidence level. If this tightest mass 
limit is confirmed, all the hybrid EOSes are ruled out but that of np+UQM. 
And, of course, the EOS of pure neutron-proton matter is allowed because it 
is stiffcr than the one of np+UQM. However, the measurement uncertainty of 
PSR J0751+1807 is not yet small enough to draw any firm conclusion and the 
tight constraint coming from Ter 5 I is more reliable. 

The hybrid stars with quark phases are indicated in Figure [3] with dashed 
lines, which reveal that all the stars with quark phase cores are unstable, i.e., 
those with quark cores will collapse. The direct reason is that hybrid EOS has a 
rather large discontinuity of energy caused by the first order phase transition as 
claimed in Sec. 13.11 Our result confirms those obtained in earlier works, where 
the hadronic phase has been described by different models, e.g., the relativistic 
mean field model or the microscopic many-body theory [26l [27]. Nevertheless 
Buballa et al. reported recently that by adopting another parameter set which 
was determined by a different method from here, 2SC phase was possible to 




8 




Figure 3: Neutron star mass-radius relations for pure or hybrid EOSes. Solid 
lines indicate the pure hadronic star and dashed lines are for hybrid stars. The 
dots denote the maximum masses and their coordinates are given in the paren- 
theses. Three horizontal lines represent the observational constraints from stars 
PSR 1913+16 (1.44M ,23j), Terzan 5 I (1.68M p4 ) and PSR J0751+1807 
(1.9OM [25]) respectively. 



exist in the star core within a very tiny window [28]. Baldo et al. suggested 
that the instability may be linked to the lack of confinement in the current NJL 
model [2"T]. Moreover, parameters obtained by fitting the vacuum properties 
might not be suitable at high densities. Therefore if the existence of pure quark 
cores in neutron stars was confirmed by future observations, the NJL model 
currently used should be modified. 

3.3 GRAVITATIONAL RED SHIFT 

In principle, the range of gravitational redshift predicted by an EOS should 
cover all the ever detected redshifts. Cottam et al. have reported a redshift z 
of 0.35 inferred by identifying three sets of transitions in the spectra of x-ray 
binary EXO 0748-676 [25]. The result has been confirmed by Chang et al.[3"D]. 
And it has been shown that the total error in this redshift is no more than 
5%[31]. Therefore, allowing for the error bar, it imposes a lower limit of about 
0.3325 to the maximum redshift. In Figure[H gravitational redshifts vs. masses 
of different EOSes are shown. EOSes with hyperons fail to construct stable 
neutron star satisfying the redshift of EXO 0748-676, so they are ruled out. 
Without hyperons, hybrid EOS of np+CFL is not allowed likely, while those 
stiffer, namely, of np+2SC, np+UQM and np, are permitted. 

Larger observational redshift gives more stringent constraint. Tiengo, et 
al., inferred that a redshift z=0A can be obtained by explaining the emission 
lines from 4U1700+24 with the Ne IX triplet[32j. Then the hybrid EOS of 
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Figure 4: Gravitational redshifts vs. masses for different EOSes. The lower hor- 
izontal line is the observational gravitational redshift of EOX0748-676[29l [3T] 
and the upper one is that of 4U1700+24[32]. The dots denote the maximum 
masses and their coordinates are given in the parentheses. Constraints of ob- 
servational masses (the same as in Figure [3]) are also plotted with three vertical 
lines. 

np+2SC would be ruled out by this constraint, and only the one of np+UQM is 
marginally permitted. However, due to the lack of identifications of other spec- 
tral features at z=0.4, it seems that the interpretation of z=0.012 for 4U1700+24 
is more reliable [32]. Therefore redshift of EXO 0748-676 is the most reliable con- 
straint till now. 

Combining the constraints of observational masses and redshifts of neutron 
stars, it can be summarized that EOSes of np+2SC, np+UQM and np are 
very likely to be favored. And if either the mass limit of 1.9M© of the most 
massive pulsar star PSR J0751+1807 or the very tentative interpretation of 
redshift of 4U1700+24 equal to 0.4 is confirmed, EOSes with CFL or 2SC quark 
phase are both denied, then the allowed hybrid EOS is only of np+UQM. It 
should be noted that rotation effect is not considered here because influences of 
rotation effect on static properties are negligible for the main observations [33j 
we currently discussed. 

4 SUMMARIES AND CONCLUSIONS 

We have used two realistic effective quark models, i.e., the three- flavor NJL 
model and the MQMC model, to describe the neutron star matter. For hadronic 
phase, EOSes with and without hyperons (npH and np) are both considered. 
And we have discussed the quark matter phase in normal and color supercon- 
ducting states, namely, UQM, 2SC and CFL. Then four possible hybrid EOSes 
between hadronic phase and quark dcconfmcmcnt phase are constructed. We 
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find that EOSes with hyperons should be ruled out by the observational con- 
straints from the mass of star Ter 5 I and/or the redshift of binary star EXO 
0748-676. Moreover, the hybrid EOS of np+CFL is also ruled out by the obser- 
vational redshift of EXO 0748-676. As a consequence, hybrid EOSes of np+2SC 
and np+UQM as well as pure hadronic EOS of np are most likely to be fa- 
vored. Tightest but less reliable constraint can be inferred by the mass of the 
most massive pulsar star PSR J0751+1807, and if it is confirmed the permitted 
hybrid EOS is of np+UQM only. 

Therefore, we conclude that observational constraints of neutron star could 
not rule out all the possible EOSes with quark phase though the neutron stars 
with pure quark cores are found to be unstable in our calculation. Both nor- 
mal unpaired quark state and two-flavor color superconducting state are likely 
permitted, and future observations are needed to determine which of them (or 
none) is the right ground state at this density. So the ground state of neutral 
matter at moderate densities could be in deconfined quark phase. 
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